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ABSTRACT 



The case is made for applying Multicriteria Decision Models 
(MCDM) to the Army's research and development (R&D) project 
selection process. Given the recent changes in the international 
strategic environment and the resulting congressional emphasis on 
applied research short of production, the need for improved 
efficiency in selecting Army R&D project alternatives is 
increasing. R&D project selection problems are non-trivial, with 
many stakeholders and multiple criteria for evaluating the 
alternatives to meet the organization's various objectives. 
Research on human cognition has revealed that typical notions 
about decision making are inefficient for dealing with multi- 
criteria decision problems. MCDM such as the Analytic Hierarchy 
Process, Multiattribute Utility Theory, Goal Programming, and 
Graphical Techniques are designed to support these decisions by 
formulating logically supportable choices. Each of these four 
models is described and a summary of their strengths and 
weaknesses is presented. 
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I. INTRODUCTION AND BACKGROUND 

The quality of American technology has enabled us to 
successfully deal with difficult military, conditions and 
help minimize the precious loss of life. We have given our 
men and women the very best. And they deserve it. 

President George H. Bush, 

State of the Union address, January 1991 



A. INTRODUCTION 

The President's moving words of gratitude reflect the 
enormous success of America's armed forces against Iraq, 
largely wrought from our overwhelmingly superior application 
of technologies to defeat naked aggression. His stirring 
emotion had to fill those responsible with quiet assurance 
that their efforts over the past several decades had been 
vital . 

At the same time, many in Congress were and are calling 
for deep cuts in military spending. Yet no one in the 
administration or Congress wants to dull America's 
technological edge. How can the U.S. maintain, much less 
improve its lead in the race for defense technological 
superiority under these circumstances? Whatever the answer, 
it will be multi-faceted and at least as complex as the 
bureaucracy from which it is born. 

Critical decisions in complex organizations are rarely 
easy, especially those involving the expenditure of 
significant public funds and with potential consequences 
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affecting the very lives of the men and women of our 
nation's armed forces. The decisions about which 
technologies are to comprise the Department of the Army's 
technology base investment strategy are central to this 
serious issue. 

Given these changes in U.S. budget priorities as well as 
national and global political and economic realities, there 
is certain to be a call for heightened oversight and 
accountability of the technology investment process. Thus, 
in spite of defense management cries to reduce 
micromanagement, scrutiny is sure to be more strenuous as 
the reverse is less politically attractive or rewarding. 

With increased scrutiny of the decision process will 
come greater need for clear logic, consistency, and 
credibility in technology investment decisions without 
stifling creativity from heightened risk aversion. Analytic 
decision models (ADM) , designed to support the 
multidimensional decision environment of defense technology 
investment, can fill this need. 

Over the past 30 years there has been something of a 
silent explosion in model development. Yet decision models 
are not widely used by the very managers they are designed 
to support [Ref. 1] . The purpose of this study is to 
introduce Army R&D managers and their support staffs to the 
virtues and limitations of some important project selection 
analytic decision models (ADM) . Given the need and their 
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effectiveness, ADM's will have an ever increasing role in 
obtaining logical solutions to problems of competing 
priorities, goals and objectives. 

B. DISCUSSION 

Analytic decision models such as the Decision Matrix, 
Hurwicz criterion, and Analytical Hierarchy Process are used 
to structure decision making under risk and uncertainty and 
may offer some support for the Army's research and 
development (R&D) /technology base investment strategy 
decision process. First of all, formal decision models help 
to formulate a structure for all issues bearing on the 
problem. They offer a rational process for assigning values 
to the important factors on the basis of current priorities 
and policy. Environmental factors such as the likely 
international political climate may also weigh into the 
problem and a "best" solution may be deduced from the 
relevant, possible alternatives. 

Research into the psychology of human judgment over the 
past several decades reveals that, while the environment in 
which decisions are made has become increasingly more 
complex, the intuitive judgmental skills of the decision 
makers has not kept pace. Rather than clarifying the 
choices, ever increasing amounts of information often lead 
to selectively "seeing only what we want to see." Filtering 
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information in this way, basing decisions predominately on 
past experience, can adversely skew results. [Ref. 2] 

Memory is another weak point in human judgment. Since 
many decisions are constructed from the data, based upon a 
reconstruction of past similar situations, limited memory is 
a source of potential human error. Additionally, people do 
not have good success judging probabilistic outcomes 
regarding risk and uncertainty. People do not have innate 
intuitive abilities to assess probable outcomes from random 
input data. Formalized decision models can provide a 
framework to organize input data which exposes these and 
other sources of potential error, systematically minimizing 
risk to improve results. [Ref. 3] 

Perhaps the most important reason for incorporating ADMs 
into the Army technology investment process is money. The 
Department of Defense (DoD) annual budget will shrink each 
year as a result of the President's agreement with the 
Congress at the close of 1990. This trend is accelerating as 
the apparent international military threat continues to 
diminish and domestic economic needs mount. The Army will 
have fewer dollars to spread around and R&D investment 
decisions will undoubtedly become more critical in terms of 
choosing "the right" programs. The current budget climate is 
likely to intensify congressional scrutiny and program 
manager accountability. 



4 



Steps to formalize the Army's technology investment 
decision making could be taken at any point in the systems 
research, development and acquisition (RD&A) process, but 
should first be applied to the initial R&D investment 
decisions to realize the greatest benefit. Given that the 
Army's Fiscal Year (FY) 1992 investment in basic and applied 
R&D will represent 14.4 percent of the Department of 
Defense's $40 billion research and development budget, even 
small improvements in effectiveness will yield substantial 
expenditure results. [Ref. 4] 

A formalized decision process using ADMs can structure 
the decision process to reveal the logical steps used to 
arrive at an investment choice. Such procedures do not 
replace the decision maker. They provide a record of the 
reasoning used to arrive at a decision. ADMs can incorporate 
current priorities and readily adapt to ever changing 
priorities, both planned and unplanned, funding decisions, 
lessons learned, and shared experience from other programs 
related to the current decision. This can ensure that no 
single factor or factors are unduly weighted or ignored in 
the decision and that decision maker bias is minimized. 

Employing a formal decision procedure using decision 
models may add stability and continuity, thus reassuring the 
decision maker and his superiors. One of the findings of the 
1986 Packard Commission and others assessing the failures of 
the defense acquisition process has been the continual 
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turnover in defense programs, resulting in inconsistent 
management and program turmoil. In his 1988 book, 

The Defense Management Challenge: Weapons Acquisition , 

Ronald Fox states that, 

In the late 1980 's, military rotation and individual 
military career considerations caused program managers 
to be assigned to programs for an average period of 
only thirty months, and the turnover is often higher 
one or two levels below the program manager. [Ref. 5, 
pp. 177-178] 

Use of ADMs could help to minimize the turmoil caused by 
personnel turnover by giving incoming managers awareness of 
the logical progression of the choices that brought the 
program office to its present position. 

Once incorporated in the overall decision process, ADMs 
could also lower the management learning curve aiding 
continuity and further diminishing turmoil and confusion. As 
a potential source of increased efficiency, ADMs might 
reduce the perceived need for micromanagement by increasing 
management confidence at all levels to which it was applied. 

These are only some of the inefficiencies that ADMs were 
designed to address. If resolving these inefficiencies is 
truly a central goal of the present defense acquisition 
process reform, formalizing the investment decision 
procedures using analytic or economic decision models may be 
one small step toward accomplishing a major part of that 
goal. In any case, helping the Army's R&D investment 
managers consistently minimize the risk and uncertainty of 
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their program choices may prove incentive enough for the 
adoption of ADMs . 

C. SCOPE OF THE THESIS 

The aim of the study is to develop a basic understanding 
of what decision support models are supposed to do for the 
decision maker, their usefulness, and limitations for 
research and development project selection decisions. The 
research study will begin by describing the basic process of 
initiating research and development projects and programs in 
the buying commands of the Department of the Army (DA) . 
Rather than examining how the decisions get approved in the 
budget submission and Program Objective Memorandum process, 
this work looks at how the new program initiation decisions 
are made at the lowest level, where operational concerns 
have some degree of priority over budgets and bureaucracy. 

The focus is on the application of ADMs for project 
selection decisions in the area of basic and applied 
research. It is hoped that the work will provide a single 
reference point for the beginning user on the availability, 
usefulness, application, and limitations of proven decision 
support models for project selection problems. 

The goal is to assess currently successful applied 
economic and analytical decision models for use by the 
Army's RD&A community. It is designed to aid those within 
the Department of the Army (DA) charged with seeking 
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technology initiatives and promising, innovative, high 
payoff opportunities to support the transition from theory 
to application. 

Currently the Army's technology investment program 
decisions are predominantly made on the basis of the Army's 
Staff Study procedure, whereby all viable alternatives are 
presented to the decision maker with their apparent 
advantages and disadvantages, based upon stated or implied 
criteria, with a recommendation for the "best" possible 
choice. The techniques introduced in this research provide 
those who must prepare such studies with a model for 
structuring each alternative's advantages, disadvantages and 
criteria in order to assess probable outcomes upon which the 
decision maker can logically evaluate the choices with more 
than intuition alone. 

D . METHODOLOGY 

This study presents information collected through a 
search of the current literature regarding DA policies for 
program/project initiation procedures to describe the 
current technology investment decision process in the 
context of the Department of the Army's overall research, 
development, and acquisition cycle. This information is 
supplemented by telephone or personal interviews with the 
authors of the Army Technology Base Master Plan ; the Deputy 
for Technology and Assessment, Office of the Assistant 
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Secretary of the Army (R,D&A) , and follow up with 
representatives of the specific buying commands within the 
Army Materiel Command, such as, the U.S. Army Missile 
Command (MICOM) , Tank and Automotive Command (TACOM) , 
Aviation Systems Command (AVSCOM) and others. [Ref. 6] 

The literature search provided some of the background 
for an analysis of the application of analytic decision 
models based on the types of decisions being made and the 
factors typically bearing on relevant decision issues. 
Subsequent analysis then details four prevalent analytic 
decision models for multiple criteria problems that have a 
significant record as effective aids to decision-making in 
fields closely if not directly related to Army research, 
development and acquisition (RD&A) program/project selection 
activities . 

The data for this analysis will be derived from a review 
of the works on the subject as well as personal interviews 
with professors at the Naval Postgraduate School and other 
experts in the field of operations research. This report is 
not a lengthy scientific investigation into the worthiness, 
efficiency, or absolute effectiveness of each analytic 
decision model uncovered in the course of this research. 
Instead, it presents scholarly opinion and advocacy of a 
particular model's usefulness, situation applicability, 
limitations, pitfalls, and/or any assumptions of those 
experts who have used and/or developed subject models based 
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on both their written works as well as personal or telephone 
interviews . 

The aim is to develop clear and sufficient evidence and 
examples to allow managers of the Army's research and 
development investment strategy and their staffs to decide 
with reasonable confidence the usefulness of a particular 
decision model to their circumstances. Each model catalogued 
will be sufficiently detailed to permit the user to examine 
and articulate the logic of the model's structure and 
application . 
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II. THE ARMY RESEARCH AND DEVELOPMENT INVESTMENT STRATEGY 

...with limited resources, an ever widening range of 
technology opportunities, rising international tech- 
nological competition, and an increasingly more capable 
third world threat, the Army must invest where the 
warfighting requirements and potential improvements are 
greatest. [Ref. 6] 



A. INTRODUCTION 

There are certainly many reasons to consider improving 
the efficiency and effectiveness of Army R&D investment 
decisions. This chapter will explore the two central to our 
discussion thus far. First, with reductions in the perceived 
threat to American security abroad and the accompanying 
reduction in defense spending, some in Congress have 
expressed serious concerns about assuring America's 
international preeminence in advanced defense technologies 
development and assuring the maintenance of the defense 
industrial base. The opening quotation serves to sum up the 
Army's understanding of these concerns. 

Beginning with Public Law 101-189, November 29, 1989, 
Congress directed the Secretary of Defense (SecDef) to 
submit an annual plan for developing those technologies 
considered most critical to U.S. defense qualitative 
superiority. This in turn led to the Army's development of 
its annual Technology Base Master Plan (TBMP) to incorporate 
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the guidance from the Secretary's Defense Critical 



Technologies Plan . The Army TBMP provides the basis for the 
way in which R&D decisions are to be made in the Army in 
light of these congressional concerns. [Ref. 7] 

Secondly, as described in the opening background 
discussion, decision theory provides a sound basis for using 
decision models in support of multiple criteria decision 
problems such as those posed by the Army's R&D investment 
process. These models may provide a logical foundation for 
structuring the analysis by incorporating ADMs for specific 
decision alternative choices. Thus, this chapter describes 
the Army's research and development investment decision 
process, followed in Chapter III by an explanation of basic 
decision making theory. 

B. THE TECHNOLOGY BASE INVESTMENT STRATEGY 

The decision process whereby the Department of the Army 
(DA) determines which key emerging technologies will be 
funded for possible concept development is a critical step 
in shaping the material warfighting capabilities of the 
future Army. The decision to fund or shelve an emerging 
technology can have far reaching implications for those 
firms focused on defense applications as well as society at 
large if the technology had proven useful in civilian 
applications. Another concern is that a critical link in our 
nation's chain of defense might be chopped from the Defense 
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Program Objective Memorandum (POM) because of the budget 
cutter's ax. 

The hard choices of what flies and what dies is sure to 
leave many defense contractors gasping their final breath in 
the 1990s. Should DA's decisions to fund certain development 
programs be based solely on the needs of our ground forces 
to overcome a foreign threat? Or, should the Army's 
leadership seek to include the broader goals of supporting a 
strong industrial base ready and able to deliver the 
American-made war goods needed for the next theater of 
conflict as well as the broad socioeconomic legislation all 
government procurement is pledged to support . The issue of 
supporting the technology base has already been partially 
addressed in Congress and the Army Acquisition Executive, 
Stephen K. Conver has acknowledged that goal. [Ref. 8] 

One thing is certain: as long as the Army's Procurement 
budget continues to shrink from $14.2 billion in 1991, down 
to $11.6 billion in 1992, and projected under $9.5 billion 
for 1993, the choices will become more difficult and the 
consequences of an inefficient choice more severe. For, in 
strictly economic terms, less Defense Procurement spending 
means less diversification over fewer systems and less 
private research and development spending by contractors 
strapped by reductions in Defense Procurement spending. 

[Ref. 9] 
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The shrinking Army budget for fiscal years '92 and '93 
will have a serious impact on Army R&D investment. While the 
Army's investment in basic and exploratory research is 
growing in response to Congressional concerns, compared to 
the rest of DoD, the Army's share is declining. For example, 
while the Army's share of the total defense budget for 
Fiscal Year '93 is projected to be just under 25 percent, 
the Army share of the total DoD RDT&E program is projected 
to decline from 15.5 to 14.3 percent. A credible technology 
research investment strategy will be essential to assure a 
viable technology base for future Army systems and could be 
a catalyst to reverse this trend. These factors increase the 
Army's technology risk in several ways. [Ref. 4] 

One risk is the shrinking industrial and technology 
base. With declining defense spending spread over fewer 
program new-starts, defense contractor funded basic research 
is expected to fall. Fewer choices between contractors and 
programs will mean that some key prime contractors may leave 
the field and with them may go critical expertise. This 
concern is perhaps even more acute for the lower tiered sub- 
contractors who support the primes with precision science 
and engineering skills. [Ref. 10] 

We learned in the 70s and 80s that limited deployment 
times meant " come-as -you-are-war “ with no time to gear up 
industry to overcome our material shortcomings. The 
alternative may be to rely on the capabilities of allies for 
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support in those areas in which we fall behind. Obviously 
this is in part what Congress hopes to avoid by supporting 
the Defense Critical Technologies Plan . 

In sum, there may not be time to recover in a crisis. 
There may not be the expertise to recover. We may not have 
the resources to commit or divert to such a recovery effort 
in time to meet the need. Reliance on allies leaves us 
vulnerable in other ways. There is, as there has always 
been, the potential that some important breakthroughs will 
be overlooked entirely in the decision process, but this 
risk increases as fewer dollars are spent on fewer programs. 

Before suggesting some ways to minimize these potential 
risks in the decision process, it is appropriate to review 
the way in which the Army selects research technologies in 
which to invest will be presented. Although the entire 
process is fairly complicated with many decision makers at 
various levels, the following is a basic summary. 

C. THE ARMY RESEARCH AND DEVELOPMENT INVESTMENT PROCESS 

The Army has divided its research program into four 
commands: the Army Materiel Command (AMC) , the Deputy Chief 
of Staff for Personnel (DCSPER) , the Surgeon General, and 
the U.S. Army Corps of Engineers. Each has responsibility 
for developing technologies critical to their area of 
expertise and each has a structure for managing various 
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aspects of the technology base to meet the needs of their 
user community. [Ref. 11] 

For example, AMC includes Laboratory Systems Command 
(LABCOM) with leadership over seven Army Research, 
Development and Engineering Centers (RDEC) and the Army 
Research Office (ARO) . The Army laboratory structure 
includes 42 laboratories and RDECs, complicating 
communication and technology transfer. A LABCOM plan for 
restructuring the Army laboratory system, called LAB-21, has 
been included in the 1990 Base Closure and Realignment Act. 
It is a six year program designed to streamline and increase 
the productivity and efficiency of the Army's research and 
development organizations. [Ref. 12] 

The ARO on the other hand, interacts with the nation's 
research universities, funding basic research in university 
laboratories throughout the country. Any of these actors can 
play a part in the process that determines which 
technologies should be funded for further development in 
this highly decentralized management structure. 

The formal decision process also includes the "user 
community" represented by the Army's Training and Doctrine 
Command (TRADOC) which has leadership over each of the Army 
combat training branches, from Artillery, Aviation, Armor, 
to the Ordnance Corps, the Signal Corps, and Medical Corps. 
Each of TRADOC ' s combat and combat support training schools 
publishes their annual Mission Area Analysis (MAA) in which 
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they identify their greatest combat survivability and 
offensive capabilities needs based on TRADOC ' s warfighting 
doctrine for the Army and the threat capabilities 
assessment . 

TRADOC then consolidates and prioritizes these needs and 
hands them off to the Secretary of the Army. The Army's 
Joint Requirements Oversight Council, chaired by Army 
Acquisition Executive, Steven Conver, Assistant Secretary of 
the Army (Research, Development, and Acquisition) marries 
requirements with capabilities and funding. The result is 
the Army's Technology Base Master Plan (TBMP) which includes 
their Technology Base Investment Strategy. [Ref. 13] 

According to the 1990 Army TBMP, thirteen areas are 
considered critical to the Army's future warfighting 
capabilities. They are called the Army ' s key emerging 
technologies. The list includes directed energy weapons, 
biotechnology, artificial intelligence, neuroscience, 
robotics, and others. Each of the thirteen "were reviewed by 
Army technical managers, scientists, and engineers. Then 
Army leadership approved the final list." [Ref. 5] 

D . THE ARMY TECHNOLOGY BASE MASTER PLAN 

The Army TBMP represents the Army leadership's top down 
guidance, objectives, and goals and includes the Technology 
Base Investment Strategy. It is intended to 
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...create an atmosphere that fosters technology 
initiatives and the pursuit of promising innovative 
opportunities while providing sufficient flexibility and 
funding to laboratory and RDEC Directors to seize local, 
high payoff opportunities. [Ref. 6, p. 1-10] 

The process of review and approval of the future 
technology investment needs, from the MAAs to the TBMP, is 
based on the experience, judgment, and intuition of the 
decision makers involved. It is an annual process, beginning 
with the RDEC Directors and their supporting staffs, up to 
the Assistant Secretary of the Army for Research, 
Development, and Acquisition (RD&A) . How the exact process 
actually works is not clearly spelled out, but one source 
summarizes it this way: 

In the case of advanced technology transition demonstra- 
tions, the Army employs a Senior Advisory Group (SAG) co- 
chaired by the Deputy Assistant Secretary for Research and 
Technology, DASA (RDA) , and the Assistant Deputy Chief of 
Staff for Operations and Plans, Force Development, HQDA. 
Submissions are made to a working group that is co-chaired 
by the Army Materiel Command Deputy Chief of Staff for 
Technology Planning and Management, and the Assistant Deputy 
Chief of Staff for Combat Developments, TRADOC . The working 
group reviews candidate proposals and provides recommen- 
dations to the SAG for new start approval. This procedure 
began in April 1990 in response to the Office of the 
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Secretary of Defense (OSD) Critical Technologies Plan (CTP) 
guidance. [Ref. 14] 

In addition to its internal process of approval, all 
Army programs have to be funded in the annual Program 
Objectives Memorandum (POM) approved by the Secretary of 
Defence and the Joint Chiefs of Staff. This step, POM 
approval, is the juncture where a perceived need becomes a 
budgeted requirement. It is possible that a critical Army 
need (in the eyes of the users) could go unbudgeted in the 
Army POM, conceivably leaving our forces vulnerable in a 
critical combat area. 

In addition to the POM, OSD publishes its annual Criti- 
cal Technologies Plan for Congress . The Services are 
expected to follow it in designing their TBMP's. The CTP 
represents the Defense Department's judgment of the 20 most 
important weapons-related technologies of the 200 or so 
technologies recommended by the Services. Multiple criteria 
are used to determine which technologies make the CTP. 

Finally, priorities are assigned based on the judgments 
of a working group with inputs from a number of defense 
industry contractors. The final decision is made by a senior 
committee representing individuals in the DoD and the 
Department of Energy with management responsibility over the 
National Science and Technology program. [Ref. 7] 

This is a decision process steeped in the give and take 
of group dynamics, political coalition building, consensus 
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and compromise among key stakeholders, their advocates and 
adversaries . There are many actors and plenty of room for 
inefficiency, if not outright error. The next chapter will 
discuss additional sources of potential error in decision 
making that can introduce inefficiencies of their own. 



20 



III. BASIC DECISION THEORY CONCEPTS 



A. INTRODUCTION 

Our goal thus far in reviewing the Department of the 
Army technology investment strategy has been to demonstrate 
that the process is not immune to the risks of human 
decision making and their resulting inefficiencies. This 
chapter will be a presentation of basic decision theory 
fundamental to an understanding of the importance of 
decision model application. The discussion will conclude 
with an evaluation of the applicability of incorporating 
analytic decision models into the DA process for potential 
improvements in efficiency and effectiveness. 

It should be noted that although there are many human 
dynamics theories and models (such as the rational policy 
model, the organizational process model, the bureaucratic 
politics model, among others) affecting the process thus 
described, their assessment is outside the scope of this 
current work. While these human dynamics models may be 
essential to the implementation of organizational decisions, 
they have little to do with analyzing and achieving economic 
efficiency . 
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B. THE ROOTS OF ANALYTIC DECISION THEORY 



The study of decision making under uncertainty has a 
long past. Beginning in the 1600's with the founders of 
modern probability theory, Fermat and Pascal, it was shown 
that outcomes over time could be quantified by the 
likelihood of their occurrence. This probability of 
occurrence times its known payoff can be summed to support a 
choice on the basis of the outcome's expected value, which 
is the average of all the potential payoffs. This idea led 
to the belief that if we can quantify the payoffs and their 
likelihood of occurrence, rational decision makers could use 
this information in judgments involving uncertainty. 

[Ref .15] 

Decision making, however, is much more complex. The 
complexity of rational decision making under uncertainty is 
supported by a vast field of research on the psychological 
profile of decision makers and spans the topics of 
heuristics and biases, prediction theory, inductive 
reasoning, causal schemata, conservatism, and risk 
perceptions, to name just some of the broad categories. 

Probably the most important work regarding the way 
managers make decisions is being done by Dr. Herbert A. 
Simon, professor of Computer Science and Psychology at 
Carnegie-Mellon University. Professor Simon's work on how we 
make decisions in organizations spans more than 40 years and 
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earned him the Nobel Prize in 1978 for his concepts of 
"objective, subjective, and bounded rationality." [Ref. 16] 
Basically, Professor Simon logically developed the 
notions of rational decision making and pointed out many 
important shortcomings in human cognitive abilities that 
limit our ability to make rational decisions. For example, 
according to Professor Simon, people; 

• tend to limit the scope of the decision problem, the 
factors impacting the choices, the alternative pros and 
cons, and objective criteria in some way to 'bound' the 
issue within their capacity and experience, discounting 
data that doesn't conform to closely held beliefs. 

• do not seek to optimize in their decision-making process 
but, rather seek to select solutions that are good 
enough . 

• attempt to construct means-ends (or cause-effect) chains 
to solve complex problems in a series of steps to 
achieve the ultimate ends in a hierarchical fashion. 
However, the conscious integration of weighing values 
against alternatives often becomes an increasingly 
complex chain and is seldom attained in actual behavior. 

• judge alternative strategies based on their experience 
and prior opinions about the consequences that follow on 
their choice of action. Obviously, no one can directly 
know the consequences of all behaviors. So, the quality 
of any decision depends on how close their experience 
corresponds to the choice problem. 

• often weigh information received initially and lastly 
more heavily than information received in the middle of 
a discussion on alternatives about which they must 
formulate a decision. 

• tend to classify problems as programmed and non- 
programmed. In organizations the former tends to drive 
out the latter. Non-programmed decisions are generally 
solved by reducing them to a series of programmed 
decisions. [Ref. 17] 
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One distinguished source on Simon's work made the 
following observation: 

... it is essentially impossible to attain objective 
rationality. To do so the decision-maker must know all 
alternatives, construct all appropriate means-ends 
chains, assess expectations for all uncertain 
consequences, etc. It is here that Simon's notion of 
bounded rationality takes form. The decision-maker can 
know these important aspects of a decision only within 
the limits imposed by experience, knowledge, time and 
effort available to search and study the decision 
environment. [Ref 16, p. 39] 

So where, one might ask, does that leave the rest of us 

mere mortals? The answer lies in Simon's notion of 

subjective rationality. For a decision to be subjectively 

rational, it must: 

• be the best choice dictated by the current state of 
knowledge using all available data 

• be congruent with the values, goals, and objectives of 
the organization with regard to the decision criteria 

• conform to the organization's subject experts' 
collective judgment regarding alternative consequence 
outcomes. [Ref. 16, p. 40] 



None of this may be particularly earth shattering to 
today's manager, but Professor Simon's 1977 book, The New 
Science of Management Decision [Ref. 17], brought many of 
these interdisciplinary concepts of decision theory together 
for the first time. It not only identified the basic human 
decision process and its foibles, but also provided some 
exceptional direction for resolving non-programmed decisions 
in organizations. It's an exceptionally positive book, in 
that regard. It should be required reading for any manager 
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preparing to confront multiple criteria decision problems. 
Its 168 pages offer extensive management insight that might 
otherwise take years of experience to attain. 

C. RISK CONCEPTS IN DECISION MAKING 

No discussion of the basics of decision theory is 
complete without a comment regarding risk. After all, no 
choice among competing alternatives would be difficult if 
everyone knew with certainty the outcomes of each 
alternative. The decision maker (DM) would choose the 
alternative whose outcome best achieved the organization's 
goals, assuming the DM were rational. The above discussion 
of objective and subjective rationality sheds light on the 
ways in which DMs deal with uncertainty in complex problems 
given human cognitive limits. Risk management is another 
important aspect of the R&D project selection decisions. 

The concept of risk is closely related to the notion of 
uncertainty. In terms of decision making, risk is the degree 
of certainty that an undesirable outcome will occur as the 
result of an action or event. To clarify this from the 
common notion of risk, think of it this way. We could say, 
for example that the risk of cancer among white males in the 
U.S. is 25 percent, meaning that roughly 25 percent of all 
white males will contract cancer in their lifetime. The 
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degree of uncertainty determines the level of risk. If we 
know an outcome is certain to result from an action, there 
is no risk associated with the decision. 

The confusion comes when we think of the popular notion 
of taking action which reduces the uncertainty of a negative 
outcome as placing us at a "higher risk." To continue the 
cancer analogy, those who smoke all their lives actually 
lower the uncertainty that they will become cancer victims, 
yet we say they are at higher risk. In fact, we can predict 
with reasonable certainty that they will indeed contract 
cancer. This popular notion of risk actually shifts from an 
event uncertainty to a timing and security uncertainty 
involved with perception and fear. [Ref. 18] 

In project selection, the risk of an alternative is 
weighed against a subjective appraisal that its payoff will 
exceed the resources needed to achieve the anticipated 
result. The appraisal must include the concept of 
opportunity cost (the payoffs foregone from the most 
promising alternative opportunities) to be valid. The 
greater the potential payoff of an alternative, the greater 
the acceptable risk, within the resource limits imposed. 

[Ref. 19] 

Even the novice DM is basically familiar with this 
concept of risk in choosing among alternatives. 

Nevertheless, the impact of risks and their effects on 
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choice cannot be dismissed. Particularly in austere budget 
climates when risk aversion is likely to increase. 

Risk averse behavior is a judgment bias that can 
severely limit innovation in the technology investment 
process and must be guarded against as any bias should be in 
rational decision making. Some of the decision models that 
will be examined in the upcoming chapters are specifically 
designed to highlight risk levels to help the DM avoid the 
inefficiency resulting from risk averse behaviors. 

D. THE DECISION ENVIRONMENT 

One final concept regarding complex decisions is the 
notion of the decision environment encompassing the 
organization making the decision. Fundamental changes in an 
organization's internal and/or external environment can 
dramatically alter the organization's mandates, goals and 
obj ectives-the basis on which important decisions rest. In 
the case of the Army's R&D technology investment decisions, 
environment refers to the national and international 
strategic, economic, and geo-political setting. As one 
research scientist pointed out in this regard, "The decision 
space is Markovian: Given today, the past and future are 
independent [Ref. 20]." 

The point is, as the Russian statistician, Markov, 
theorized, the future state of nature depends only on the 
present environment and cannot be accurately ascertained 
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from the past because the future is independent of past 
events. They are connected only by a sequence of events in 
time. Because the outcome payoff of a course of action is 
directly linked to the environment in which it results, the 
DM's payoff assessment must incorporate his belief of the 
future state of nature at the time that the program achieves 
maturity. [Ref. 20] 

This notion is especially pertinent to a development 
cycle conservatively estimated to take 20 years from concept 
to fielding. It is this notion of Markovian independence 
that has given operations research its broad footing and 
support for the development of decision models. No other 
discipline copes with this concept in a decision context. 

E. DECISION THEORY SUMMARY 

Risk, payoff, uncertainty, rationality, opportunity 
cost, judgment, all relate to statistical probabilities in 
choosing among outcomes, which can leave all but the 
operations research analyst more than a little nauseous. The 
point in providing decision theory is to help Army R&D 
managers make better, informed choices supported by 
technical analysis in their organizations, not to be better 
statisticians . 

Even if no decision models are used to aid the decision 
process, keeping human cognitive errors in mind can improve 
decision quality. Models were never developed to replace 
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human judgment but to enhance its capacity. The following 
judgment biases and pitfalls are discussed in chapter I: 

• filtering bias 

• experience bias 

• to give priority to information by order received 

• memory capacity limitation 

• conformity to prior beliefs 

• causal chains of likely outcomes limitation 

• risk preference bias 

• independence of the future from the present 
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IV. INTRODUCTION TO ANALYTIC DECISION MODELS 



. . . when asked to advise companies on the acquisition 
of a computer, my advice was, before any commitment, 
make a careful determination of its need and how they 
would use it. I soon realized that this was poor advice- 
a company only acquired the ability to make sound deci- 
sions about computers by hands-on experience with them. 

The first investment in a computer was not to be judged 
by its cost saving potential-it might have none-but by 
its contribution to intelligence capabilities and subse- 
quent decisions. 

Dr. Herbert A. Simon, 1977 

The New Science of Management Decision , p. 43 
Dr. Simon's comment regarding his early advice to 
companies' attainment of increased "intelligence 
capabilities" can apply equally well today to multi-criteria 
decision support models. These two powerful management 
tools, decision models and computers, have a nearly parallel 
development history. Decision support models of any 
complexity are almost completely reliant upon computers for 
their often lengthy computations . This can at once serve to 
enhance a model's considerable utility and further shroud 
its seemingly mystical results. 

Multicriteria decision models (MCDM) can confound the 
manager confronted with dealing with the models' results and 
the analysts' technical interpretation. Although they offer 
the manager improved leverage in enhanced decision making 
capabilities, it has been widely documented that management 
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places minimal emphasis on their use in actual decision 

making [Ref. 1 and 21] . Yet, as one important source put it: 

Whether a modeler's recommendations are accepted or 
rejected, they must be questioned and understood. If 
they are rejected, equally quantitative arguments may be 
required to contradict the analysis.... The planner who 
uses computers and formulas must be reckoned with. [Ref. 
22, p.l] 

A. BACKGROUND ON GENERAL DECISION SUPPORT MODELS 

Although many decision support models have been 
developed since management science got its start after World 
War II, many are not well suited for R&D project selection 
problems with multiple objectives. For example, Professor 
William Souder of the University of Alabama, Huntsville, in 
his 1984 book, Project Selection and Economic Appraisal 
[Ref. 23], summarizes nearly the entire field in 150 pages. 
However, the models he presents are those which primarily 
focus on one obiective -the project's economic benefit or 
payoff . 

Economic decision models such as cost/benefit analysis, 
break-even point, make or buy, return on investment, payback 
period, and internal rate of return are all excellent and 
may be absolutely essential to business decisions where 

profits determine performance (if not out right survival). 

« 

Others, like the Simplex method and Dynamic Programming, use 
a sequential approach to optimize resource allocation. By 
allocating the scarce resource in question (personnel, 
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space, machine time, funding, etc.) to the candidate 
projects, as many of the candidate projects (or jobs) are 
completed as the resource allows. 

Both single objective and sequential decision support 
models require that alternative action outcomes be known 
with mathematical certainty and measurable in terms of the 
subject resource (usually in dollars) . In the public sector, 
this is frequently not the case. Many other factors weigh 
into the problem in a free and democratic society from many 
quarters and competing stakeholders. From benefits to the 
local community to the nation's technology base, in terms of 
jobs and welfare, to effects on the environment, to issues 
of keeping annual program budgets fully committed - many 
issues must be simultaneously addressed for a decision model 
to be completely beneficial to the user. (See, References 
17, 22, 23 and 24, if need be, for an excellent development 
of the economic and analytic decision models mentioned 
here) . 

Obviously, if the choices between alternatives were 
trivial there would be no need for detailed modeling of the 
decision problem. If the DM is reasonably certain of the 
outcome of each alternative project available and suitable 
to comprise his R&D program in a future fiscal period, the 
individual's intuition and judgment alone are sufficient. 
However, most R&D project selection decisions are complex 
for several reasons . 
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They often involve a host of interrelated elements with 
varying degrees of impact on the decision. Furthermore, the 
experts upon whose judgments the DM may have to rely, will 
often be stakeholders in the decision's outcome, with their 
own varied priorities weighing in their decision criteria 
[Ref . 26] . 

Expert input is valued, but may need to be standardized 
or quantified to weigh considerations on a level basis, to 
avoid lending undue credence to any particular point of view 
[Ref. 26]. Essentially, if the trusted experts disagree on 
the criteria and approach for weighing the value of 
alternatives, and the outcome is critical, the decision is 
non-trivial and the need to quantify qualitative 
considerations and factors to structure the decision problem 
should be apparent . 

B. THE MULTI -OBJECTIVE ENVIRONMENT OF R&D PROJECT SELECTION 

Consider for a moment the Army's Research, Development 
and Engineering Center (RDEC) directors who are expected to 
balance the many objectives listed in Table 4.1 [Ref. 6] 
from Congress, Office of the Secretary of Defense (OSD), on 
down to Army Materiel Command priorities. In addition, there 
are a number of technology investment objectives that 
conflict or complement these stated goals. Some current 
examples include minimizing development time, life-cycle 
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cost, damage to the environment, while at the same time 
maximizing dual use technologies, performance capability, 



TABLE 4.1 THE ARMY TECHNOLOGY BASE 
STRATEGIES; Source: 1990 Army 
Technology Base Master Plan [Ref. 6, pp. 
1-9 - 1-10] 



*Enhance warfighting 

*Balance R&D between near, mid, and 
far-term needs 

*Balance between next generation, 
emerging technologies, systemic 
issues, and support capabilities 

*Seize technology initiatives 

*Leverage R&D outside the Army 

*Reduce concept risk and development 
times 

*Foster innovation and seek high 
payoff opportunities 



and protecting the industrial and technology bases, all 
within a projected budget limitation essentially beyond his 
or her immediate control. 

This is the decision environment multicriteria decision 
models (MCDM) were developed to handle and support. It is an 
environment in which the decision maker (DM) must achieve 
multiple goals with a given budget over a broad range of 
continually changing R&D project alternative opportunities, 
each with its own expected returns for achieving at least 
some of the required objectives. 
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How can project alternatives be compared to determine 
which ones best satisfy program decision criteria? Where a 
considerable budget is involved with frequently changing 
goals, objectives, criteria and priorities, the decision 
process will hardly be routine. As mentioned previously, 
with increased scrutiny and oversight, expert opinion alone 
may no longer suffice. 

The basic answer to our "How to compare" question lies 
in quantifying subjective judgments from experts in a 
systematic, formal analysis of alternatives and objectives. 
Some might conclude that complex issues in the public sector 
should be left to subjective evaluations, but for many of 
the reasons stated in the second chapter, it is quite often 
inefficient to do so. 

For example, although we may not be able to directly 
quantify the value of several aircraft designs of near equal 
cost- some offering improved performance, others increased 
efficiency, and still others technological advantages- we 
could ask experts to judge which design (s) is preferred over 
the other and use the preferences to develop a rank 
ordering. Quantifying the rankings and comparing them 
against priorities and constraints can lead to a logical 
choice among alternatives. [Ref. 27] 

In conclusion, several important multi-criteria decision 
models (MCDM) have been developed to support the kinds of 
decisions facing the Army in its R&D project selection 
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process. These are non-trivial decisions involving extensive 
cost, payoff data, and opinion as well as many competing 
goals, objectives, criteria and priorities. By organizing 
the input data, quantifying expert judgments, and forcing 
choices among competing criteria, MCDM can help the decision 
maker (DM) overcome many of the inefficiencies inherent to 
human decision making- namely, biases and error- discussed 
in chapter III. 

C. PURPOSE AND STRUCTURE OF THE UPCOMING FOUR CHAPTERS 

The next four chapters are designed for the management 
DM who may not be aware of the potential of MCDM 
applications for supporting R&D project selection decisions 
in the public sector. The models chosen for presentation are 
those MCDMs that predominate the literature of management, 
management science and operations research. As such, they 
are considered the most useful and widely recognized for R&D 
project selection and other resource allocation problems of 
this kind. These are the MCDMs which the management DM in 
the public sector is most likely to encounter and for which 
some knowledge of applications will be increasingly 
important . 

We begin with a summary of some of the graphical 
techniques for multi-criteria decision support developed 
from several sources. We will then move to a development of 
the Analytic Hierarchy Process (AHP) . Next, the 
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Multiattribute Utility Theory (MAUT) will be detailed. 
Finally, chapter VIII will present a development of the Goal 
Programming (GP) method for solving problems of choice with 
multiple criteria. 

It is important to keep in mind that this is not a 
technical development or analytical assessment. As a 
management student curious about the power and limitations 
of MCDMs, my goal is to spark the curiosity and inform my 
peers about what management science is contributing to this 
specific decision environment. As in an explanation of the 
value and power of application software, the point is what 
these tools can do for us, not the bits and bytes of how 
it's done. A deeper understanding can and should be sought 
from the primary sources listed as the references. 

Throughout the discussion so far, several terms central 
to decision support modeling have been used, the meanings of 
which may be becoming somewhat blurred. To alleviate 
confusion, the following definitions are offered to assist 
the reader in keeping each term in focus: 

• criteria: the measures of effectiveness against which 
alternatives are evaluated. 

• attributes: the variable outcomes contributing to pro- 
gram objectives. 

• objectives/goals: stated and implied R&D program 
mandates and priorities. 
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• constraints: the physical and economic limitations to 
program optimization. 

• alternatives: candidate projects for selection. 
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V. GRAPHICAL APPROACHES TO MULTI -CRITERIA DECISION PROBLEMS 



We begin our discussion of multicriteria decision models 
with a development of some graphical approaches to solving 
the Army R&D selection problem. These techniques are 
adequately covered in the literature of management science 
for solving choice dilemmas with conflicting attributes. But 
more significant is their flexibility. These models can 
incorporate data from simple cost/benefit analysis to 
sophisticated risk and utility functions if desired by the 
DM. 

First, the model will be presented through a discussion 
of the development process to present some idea of how the 
model works to turn input data into output recommendations. 
This will be followed by a discussion of the technique's 
strengths and weaknesses identified through research or 
actual applications where available. The chapter will 
conclude with further analysis of the limitations, pitfalls, 
and strengths of this technique. Some comparisons between 
graphical models and models to be developed in later 
chapters will also be provided. 
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A. GRAPHICAL MODEL STRUCTURE AND FORMULATION 



Graphical techniques are a symbolic depiction of the 
expected outcome for each alternative with respect to each 
attribute. Subject matter experts establish the measures of 
effectiveness criteria upon which each alternative is to be 
judged and establish some effectiveness rating scale for 
scoring the alternatives. For example, some objectives for 
an R&D program might be to: 

• Maximize utility and effectiveness (coverage such as 
dual use technologies, interoperability, etc.) 

• Maximize research personnel utilization 

• Minimize development time 

• Minimize resource consumption (funding, overhead, and 
support costs, for example) 

• Maximize "critical technologies" state-of-the-art 

• Minimize development risk (such as outcome suitability 
and contractor experience/ability to deliver) 

Next, some measures of effectiveness (MOE) for each 

objective would have to be established and agreed to by the 

subject experts/stakeholders. Determining the program 

objectives and adequate MOE for evaluating each alternative 

project under consideration is the first step in applying 

graphical techniques. [Ref. 28] 

Some of these measures, such as costs, are hard criteria 

for which reasonable data figures may be available. Some of 

the others are soft criteria that would have to be based on 

expert judgment, experience and intuitions of those 
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involved. These soft criteria must be quantified so that 
measures can be compared on an equivalent basis. [Ref. 29] 
One way to do this is to establish "scoring functions" 
or graphs which depict ratings for the predicted alternative 
attributes (outcomes) for each of the selected criteria, 
such as those shown in Figure 5.1. Both quantitative and 
qualitative results can thus be shown. [Ref. 30] 




Figure 5.1 Example Scoring Function Graphs 



One graphic technique for multiattribute decision 
analysis presented by Zeleney, Canada and Sullivan, and 
others, [Ref. 29, 30 and 31], is called Polar Graphing. This 
method consolidates the results from scoring function graphs 
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to visually demonstrate relative contributions of each 
alternative's attributes. Two example presentations of the 
model's output are shown in Figure 5.2 [Ref. 29 and 30]. 




Figure 5.2 Typical Polar Graphs 



Polar graphs can facilitate decision making by revealing 
those projects that make the greatest contribution to the 
RDEC research and development program objectives. As with 
many graphs, however, the axis, scale values can be 
manipulated to represent the data in different ways for a 
variety of interpretations. 

One criticism of the polar graphing technique is that in 
structuring this method of multicriteria decision modeling, 
no author has formulated a specific way to weight the 
criteria (represented by the "spokes" in Figure 5.2) for 
evaluating the attributes (depicted as the "spider web” 
linear values around the spokes) of the various alternatives 
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in a consistent way. Although general "utility values" can 
be chosen and assigned using the scoring function graphs, 
this is considered highly subjective. While the model 
analyst and DM may develop agreement on consistent MOE, 
their interpretation and the resulting axis, scale values 
might not hold up under close scrutiny. [Ref. 32] 

This issue of MOE validity for the selected utilities 
and their representation of the true values is dependent, at 
least in part, upon the specific program objectives and how 
closely the utility values and scores represent them. If the 
scale values are accurately represented, the graphs may 
reveal relative desirability between alternatives. Then, as 
long as all experts and stakeholders agree which attributes 
should dominate the selection criteria, a valid choice can 
be made. [Ref. 26] 

Looking at the example in Figure 5.2, another 
shortcoming is evident. Clearly, with more than five or six 
criteria and three to five alternatives, the aid this 
presentation provides is rather dubious. It would do little 
to clarify the choice considerations. On the other hand, 
consider the inadequacy of traditional models, such as cost- 
benefit ratios, that simplify the problem by limiting the 
analysis of alternatives to a comparison of only two 
criteria. A polar graph analysis could demonstrate the need 
for greater clarity among alternatives. [Ref. 29] 
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Another graphical method that at least partially 
overcomes this visual complexity problem is suggested in 
Keeney and Raiffa's Decision with Multiple Objectives [Ref. 
27]. Their approach depicts the attribute scores of each 
alternative in a "performance profile, " like that shown in 
Figure 5.3. In this technique, the vertical lines depicting 
the attribute scales are not standardized to an evaluation 
rating or scoring function as with polar graphing. 




However, the authors indicate that this can be done if it 
clarifies the problem. In addition, Keeney and Raiffa's work 
covers the development of weighted value functions for 
determining preferences between criteria. [Ref. 27] 

Graphic techniques used to support multi-criteria 
decision problems are useful because they are intuitive by 
design. The information presented can be used much as in a 
traditional trade-off analysis assessing more than two 
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criteria. The support staff officer can show a lot of the 
information quickly and clearly without a lot of fanfare or 
technical jargon to explain the approach. The results or 
output supporting the decision at hand can be largely self 
explanatory. This is particularly true for comparing self- 
evaluation and cross-evaluation of alternatives, as seen in 
the examples [Ref. 26] . 

B. ASSESSMENT OF THE STRENGTHS, LIMITATIONS, AND WEAKNESSES 

OF THE GRAPHICAL TECHNIQUES 

The first comment heard regarding the value of the polar 
graphs or the performance profiles is what scale values are 
to be used and how are these values determined? In both 
cases, Keeney and Raiffa's development of value functions 
and utility functions can be used [Ref. 27]. In addition, 
more familiar decision aids can also be incorporated, such 
as net present value, internal rate of return, cost-benefit 
analysis, and many of the other financial and economic 
assessment techniques. Some authors suggest adjusting these 
traditional value equations for risk and time factors to 
further enhance their usefulness and accuracy [Ref. 33]. 

This is actually an important strength of multicriteria 
graphical techniques. That is, they are extremely flexible 
as mentioned at the outset. Many of the other MCDM that will 
be developed in the upcoming chapters do not incorporate the 
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traditional economic analysis formulas which may in part 
account for the lack of use of MCDM in American business and 
industry . 

In the often cited surveys by Matthew Liberatore and 
George Titus regarding MCDM for R&D project applications, 
none of the 40 respondents from 29 "Fortune 500" firms 
queried in 1983 used MCDM in their R&D project selection 
process. However, net present value and cost-benefit 
analysis were reportedly used 74 percent and 62 percent of 
the time, respectively. Clearly, for a model to be seen as 
effective, it must first be regarded as important and 
reliable by the intended user. On this basis, the graphical 
approaches must be given high marks for user confidence. 

[Ref. 21] 

Another shortcoming cited by Vickers and Belton in their 
1988 study on graphical techniques (which they refer to as 
"visual interactive models") is the time and difficulty of 
formulating these models and their accompanying displays for 
real problems. However, the authors suggest the use of 
several commercially available software packages to assist 
DMs and analysts with formulations. They recommend Symphony 
and Framework for the novice and Why , Hocus , Witness , and 
Genetik for more advanced formulations by analysts. The 
authors also present a complete model development with 



46 



results from input by 60 student evaluators. Their work is 
exceptionally informative and helpful for the beginner . [Ref . 
31] 

Dr. Milan Zeleny of the Graduate School of Business, 
Fordam University argues effectively that polar graphs and 
other graphic techniques are more than mere visual displays 
because they serve to improve decision making. He points out 
that most MCDM ease the decision process by separating the 
problem into its sub-parts and guiding the DM to state his 
preferences in a piecemeal fashion. 

Zeleny argues that this only reorganizes the problem 
rather than resolving the inefficiencies resulting from 
bounded rationality. He explains that this is primarily 
because such methods disrupt the holistic view of the 
problem. It is the holistic view that is essential to good 
decision making and the essence of human cognitive skills. 
Graphical techniques, on the other hand, fit all the 
essential pieces of conflicting information together to 
diminish the role of the analyst/facilitator and increase 
that of the decision maker. [Ref. 29] 

In conducting this research it appeared that another 
important weakness of graphical techniques for multicriteria 
decision problems is the lack of data from users conducting 
actual problem analysis. The only data found were from 
hypothetical modeling of simple problems. This might be a 
result of the fact that no particular author or group claims 
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ownership of these methods to champion the research efforts 
necessary. Additional field work needs to be done to prove 
the theory and reassure potential users of the benefits 
obtainable . 



48 



VI. THE ANALYTIC HIERARCHY PROCESS 



Chapter VI will familiarize the reader with the Analytic 
Hierarchy Process (AHP) for aiding the DM in determining the 
optimum mix of R&D projects to comprise the program within a 
limited budget environment. Much has been written about the 
strengths and weaknesses of AHP and its applicability to 
this particular problem. This chapter will present an 
overview and description of how the model is formulated 
followed by some assessments of AHP from users of the 
technique . 

A. INTRODUCTION TO AHP 

The Analytic Hierarchy Process was developed and 
introduced by Professor Thomas L. Saaty of the Wharton 
School, University of Pennsylvania and documented in his 
1980 book of the same name [Ref. 34]. This technique is 
quite possibly the most important for quantifying 
qualitative expert opinion and judgment for decision 
support, at least in terms of its ease of use, its parallel 
to actual decision making processes, and its broad 
applicability. AHP enjoys the widest acceptance among 
managers of all the multi-criteria decision models that will 
be presented in this work. 
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This technique requires the DM to construct a hierarchy 



with the organization's top objective, broadest goal, or 
mandate at the top. Subsequent levels of the hierarchy 
include more specific objectives, activities, or effects 
necessary to achieve the top goal, such as those listed up 
in Table 4.1, from the Army's Technology Base Master Plan 
[Ref. 6]. The lower levels are the attributes of specific 
alternative project choices (courses of action) that 
contribute to the organization's objectives. The final level 
is comprised of the specific projects themselves. An example 
three tiered hierarchy is shown below in Figure 6.1. [Ref. 
30] 




Figure 6.1 Sample Decision Hierarchy 



The process of developing this decision hierarchy breaks 
the decision down into its sub-elements and is designed to 
get the organization to concentrate on where its resources 



50 



need to be focused to achieve their ultimate goals. There is 
little room for hidden agendas, nest feathering, or the 
"underling problem" (wherein a subordinate seeks to maximize 
personal aspirations regardless of their impact on the 
organization's goals) when priorities are examined in this 
process. [Ref. 22, p.43] 

The hierarchy development process can also disclose 
relations and overlap of goals and projects. It is also 
hoped that the process will help to examine justifications 
in terms of "virtues" or planning criteria of future returns 
on R&D. The hierarchy can help clarify what has been 
omitted, where boundaries have been drawn sub-optimally 
narrow, and where principles and programs extraneous to the 
organization's basic mission are being used to justify 
expenditures. [Ref. 35] 

The hierarchical structuring of the decision problem is 
to be developed to deal with three issues in the R&D 
investment problem: 

• How do we plan for research and development under a 
multitude of uncertainties? 

• How can we design and implement our R&D program so 
that, if successful in its separate parts, it will 
aggregate to a coherent system a quarter century hence? 

• How can the organization, given a quality selection of 
projects that emerge from our analysis of the above 
questions, allocate our budget cost effectively so that 
benefits of investments in technologies today will 
accrue on schedule? [Ref. 35] 
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Within the hierarchy itself, each element at a given 
level, called "virtues" (or attributes), may also be 
assessed as obstacles. As the hierarchy is developed, the 
experts can evaluate the severity of the challenges posed to 
operations if a given attribute were inadequately resourced. 
The intent is for the subject experts to identify areas 
where future funding will be needed to stave off potential 
lags in capability. This is to orient the exercise toward 
anticipating growth needs rather than simply extrapolating 
trends. [Ref. 35] 

The sub-objectives of the hierarchy can also be 
established as priorities broken out by the organization's 
divisions. The DM can use the hierarchy to assess the 
relative merits of alternative programs, projects and 
technologies. What emerges is a hierarchy of benefits that 
can be expected to accrue to each program area. The 
potential benefits can be discounted and thus used to plan 
current investment over the organization's planning horizon- 
one, five, or more years out. But this is just the first 
step in the AHP analysis . [Ref . 35] 

Once the hierarchy has been designed, the next step is 
to elicit expert opinion to compare the attributes on each 
level to one another to determine and establish the relative 
merit of each attribute in terms of its contribution to 
achieving the next higher level goal. The experts assign a 
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number value from a scale like the one shown in Table 6.1. 
These scores (called preference factors) are recorded and 
organized into a separate matrix for each level's elements. 

TABLE 6 . 1 

SAMPLE PREFERENCE SCALE 



IF attribute 1 is... 

preferred to attribute 2 

"Equally", then rank is 1 



"Slightly" = 3 
"Slightly more" = 5 
"Strongly" = 7 
"Always" = 9 



The scores are then aggregated and standardized using 
the formal theorems of matrix algebra to derive relative 
weights for the projects at the lowest level of the 
hierarchy. The experts' preferences are thus translated into 
a project selection ranking of the alternatives with respect 
to the attributes by revealing which projects are most 
adequate for achieving the overall goal. [Ref. 25] 

Although the matrix algebra calculations can get fairly 
extensive with only a few levels of attributes and experts 
scoring, software is commercially available to conduct the 
calculations. The use of a computer software package is 
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essential for quantifying the preference value judgments of 
several experts from different departments or divisions. 

[Ref. 25] 1 

Each expert with input to the decision process must 
make l/2n x (n-1) paired comparisons (where n is the number 
of elements on a level of the hierarchy) . This means that a 
hierarchy with ten alternatives, eight attributes, five 
objects, and a primary goal would require 83 paired 
comparisons from each expert. Because of this need for 
extensive input data as well as the limits of human 
cognition for making these comparisons, Professor Saaty and 
others have recommended keeping the number of elements in 
any level at no more than nine and the number of levels to 
between three and five. [Ref. 25 and 36] 

B. ASSESSMENT OF THE STRENGTHS, LIMITATIONS, AND WEAKNESSES 

OF THE AHP MODEL 

In 1984, researchers conducted a multi-year study of an 
actual application of the AHP method to the R&D selection 
process of a pharmaceuticals manufacturing firm in England. 
The application was conducted to assess the strengths and 
weaknesses of AHP against those of other multiattribute 
decision methods for an actual case. The study was not an 

1 For additional information, contact Decision Support, 
Inc. about "Expert Choice" at 4922 Ellsworth Ave., 
Pittsburgh, PA 15213 or call (412) 682-3844. 
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experiment but rather an application of AHP in the public 
sector. The problem required ranking ten alternative 
projects against eight criteria attributes selected by the 
firm's DM and his project managers. [Ref. 37] 

The study is a good example demonstrating the 
formulation process, model output, and user reactions. The 
decision maker and his staff in this case regarded the 
process of using the preference factor ranking scale 
confusing and the extensive paired comparisons, described 
earlier, as tedious. Even after explaining the preference 
factor to the users, the authors reported many still made 
errors and found that they could not remember how they had 
ranked earlier alternatives to attribute comparisons the 
further they got into the procedure. 

The researchers were able to work the participants 
through these formulation problems to obtain output the 
participants found reliable. In spite of the shortcomings 
they had cited, all of the participants found the process of 
structuring the decision in an analytic hierarchy useful for 
gaining insight and understanding of the important factors 
in research planning. In fact, the authors report in their 
conclusion that the R&D division of the firm has 
incorporated AHP, with some recommended modifications, into 
its routine R&D selection procedures. 

In a 1987 follow up to this original research work, 

Geoff Lockett and Mike Stratford found the R&D division at 
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ICI Pharmaceuticals continued using AHP without the 
technical advisors to guide them. The DM was utilizing and 
acting on the numerical results obtained from the model. The 
section managers and project manager found that formulating 
the decision problem with AHP was helpful and the output 
was useful. [Ref. 38] 

However, Lockett and Stratford also stated that some of 
the respondents (including the project manager himself) were 
continuing to misuse the preference factor scale and changed 
their input choices once they had additional clarification 
on its correct use. The authors indicated that this was a 
significant drawback to AHP since, in their opinion, "In 
practice, judgmental models will have to be used without 
technical advisors if they are to become accepted [Ref. 38, 
p. 399]." In addition, after conducting a comparison, they 
recommended incorporating multiatribute utility theory 
(MAUT) methods for choices among similar attributes to 
improve results. 

Several other improvements have also been suggested by 
researchers working with the AHP technique for the R&D 
selection problem [Ref. 39, p. 305 and p. 345] In addition, 
Professor Saaty has responded to the criticisms regarding 
the data from value comparisons using his original 
preference scale in a 1987 article in Decision Sciences . He 
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suggests ranking alternatives with a relative proportion 
method rather than his original scale values of 1-9. [Ref. 
40] 

There literally seems to be no end to the controversies 
surrounding the value and weaknesses of AHP in assisting 
decision makers to evaluate multiple alternatives with 
multiple conflicting criteria. While one source would find 
the hierarchic structure development awkward, others said 
that it helped their understanding. Some decried the lengthy 
comparison process as unnecessarily tedious, while others 
argued this was a strength of AHP through its development of 
a kind of "average preference" value. In any case, the 
overwhelming consensus in the literature is that applying 
AHP to the decision process improved the quality of 
decision making for the DM. But the method is not without 
its critics, many of whom advocate alternative MCDM methods 
of their own (see, for example. Ref. 25, 26, and 28.) 
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VII. MULTIATTRIBUTE UTILITY THEORY 



Another multicriteria decision model which has proven 
useful for supporting the evaluation of projects for R&D 
selection is multiattribute utility theory or MAUT. As has 
been the pattern thus far, the chapter is organized in two 
parts. The first will introduce the basic elements of the 
model and its formulation and the second will review the 
important literature regarding the model's strengths and 
limitations. The goal is to allow the reader to^garner some 
notion of the value of this technique for supporting 
decision problems with multiple objectives. 

A. INTRODUCTION TO MULTIATTRIBUTE UTILITY THEORY 

This important multicriteria decision model uses 
theories from mathematics and economics regarding utilities 
to quantify DM preferences. The first and dominate example 
of this technique was presented by Ralph Keeney of Woodward- 
Clyde consultants and Dr. Howard Raiffa of Harvard 
University in their 1976 book, Decisions with Multiple 
Objectives: Preferences and Value Tradeoffs . [Ref. 27] 

The Keeney-Raif f a MAUT decision model uses fairly 
advanced mathematical development and economics manipulation 
to evaluate alternatives against multiple criteria and 
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arrive at the optimal course of action to achieve the 
desired outcome, goal or objective. Although its application 
to the R&D project selection problem is not as well 
documented as that of AHP, it has been widely applied to 
many types of selection problems with multiple qualitative 
and quantitative criteria. It must therefore be included in 
any serious discussion of this class of MCDM. 

Like AHP, Multiattribute Utility Theory (MAUT) begins by 
determining the overall program goal, objectives, 
attributes, criteria, and project alternatives and 
organizing them into a hierarchy. The notion is to divide 
the multicriteria problem into its basic component parts, 
determine the DM's preferences on each component, and derive 
the alternative that best fulfills those preferences. The 
process can become fairly complicated. The following is a 
narrative explanation of the steps in the MAUT procedure. 

Once objectives, attributes, and alternatives are 
determined, subjective measures of effectiveness are derived 
and assigned to each attribute. Objective data are used for 
this whenever available and applicable. The DM's quantified 
values and/or expected values are thus established for each 
attribute. Value functions are then derived by summing the 
attribute values that represent the DM's preference 
structure by assigning a weighting constant, an exponent, or 
both to each attribute element of the function for each 
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alternative [Ref. 41]. Several example value functions 
adapted from those presented by Keeney and Raiffa are shown 
in Figure 7.1 [Ref. 27, p.81]. 



v(X) =c 1 x 1 +c 2 x 2 +c , 3 x 3 
v(Y) =y 1 2 y?y 2 

v(Z) =c 1 z 1 +c 2 z 2 +c 3 ( z 1 -Jb 1 ) 2 (z 2 -jb 2 ) 4 



Figure 7.1 Sample Value 
Functions for Hypothetical 
Alternatives X, Y, and Z 



Next, value tradeoffs are conducted systematically in 
which the DM must decide how much of one objective he is 
willing to give up to improve his chances of achieving one 
of the other objectives. Outcome probabilities are used to 
determine expected values where uncertainties are involved. 
The tradeoff analyses are conducted between the alternative 
value functions using marginal analysis as in classical 
economics. The basis for this approach is that the utility 
of any course of action under risk equals the expected value 
of its payoff. 

Multiatribute utility theory relies heavily on economic 
principles of Pareto optimal analysis. A fundamental 
assumption of this analysis is that for each act there is an 
outcome, and the set of all the available alternative 
outcomes which are most efficient comprises an efficiency 
frontier for a particular decision maker. If we, as that DM, 
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knew which acts (alternative programs) led to the most 
efficient outcomes, and we were rational, we would choose to 
do those acts to obtain the maximum benefits. That is, we 
would choose the Pareto optimal set of alternatives. With 
MAUT, the analysts use probability and DM preferences to 
derive the DM's Pareto optimal outcome set. [Ref. 27] 

The value functions and the outcome probabilities are 
combined to obtain a utility function for each alternative. 
The alternatives with the greatest utility for the DM are 
those that should comprise his R&D program. Numeric values 
can be determined for the attributes comprising the utility 
functions so that a utility value can be calculated. Thus 
the alternative projects can be rank ordered from greatest 
to least utility for achieving the organization's goals. 

[Ref. 42] 

Obviously, the mathematical expressions get rather 
lengthy with only a few attributes for each alternative. Not 
only that, but the utility functions involve the integration 
of the area under the theoretical Pareto optimal curve and 
the curves can be monotonically decreasing or nonmonotonic 
increasing in three space with additive or multiplicative 
risk values! This is not the kind of analysis the typical 
management decision maker is comfortable with. The tradeoff 
analysis process for just three alternatives takes the 
authors nine pages to explain. [Ref. 27] 



61 



Utility theory says that the DM must assign a utility to 
a course of action (project alternative) based on his 
perception of the risk of the probable outcome for each 
alternative. For example, if the most desired outcome were 
only half as likely to occur at the conclusion of project A, 
but a less desired outcome were sure to occur with 
alternative B, the DM should assign a higher utility to 
project B, and fund B accordingly. The point is, the DM 
should act to maximize his expected utility. The DM's 
perception of risk is factored in as well. If the DM is risk 
prone or risk averse, he will naturally adjust his utility 
for each alternative. [Ref. 22] 

With MAUT the assumption is that DM preferences are 
consistent and independent of each other. The idea of 
decomposing a problem implies that there will be less error 
from random judgments when the DM can focus on single 
components of a decision problem. The central assumptions 
are that the DM will make rational choices that optimize his 
utility and that a decision is rational if it maximizes the 
DM's expected utility. 

One other point unique to MAUT is that the technique 
allows for only one DM instead of inputs from multiple 
experts. Utilities of multiple stakeholders are not 
aggregated and factored in mathematically, but their 
opinions should be weighed into the DM's utility function 
for each alternative. The DM could call for subject experts 
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to provide input to set attribute values and assist with 
utility tradeoffs preferences. It would then be up to the DM 
how he would use their input to weigh on his own judgments. 
[Ref. 41] 

B. ASSESSMENT OF THE STRENGTHS, LIMITATIONS, AND WEAKNESSES 

OF THE MAOT MODEL 

First of all, do not feel intimidated if, having read 
the above explanation, you're not sure what you read. This 
seems to be a frequent response to utility theory in general 
and MAUT in particular. In many instances this was the 
typical user response. In comparisons with other techniques 
such as AHP and goal programming, the outcomes for optimal 
alternative ranking were strikingly similar. This often 
reassured users of the method's value. However, the concern 
remained that this model may be difficult to use without the 
assistance of a competent analyst to work the DM through the 
process (see, for example, Ref. 37 and 41). 

This lack of understanding of how the MAUT quantifies 
qualitative preference "data" and probable outcome functions 
to obtain its rank order solution is the complaint most 
common in application analyses uncovered in this research. 
Many of the researchers have questioned the value of MAUT on 
the basis of this concern, pointing out that the 
formulation itself- the way value and preference questions 
are themselves presented- may too strongly influence the 
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optimal outcome results. More study should be done in this 
regard, but some research has already shown that this point 
does not negate the value of MAUT to decision makers in 
actual practice. [Ref. 43] 

One of the other common complaints about MAUT is the 
lack of a means for sensitivity analysis. The value 
functions and probability analysis leading to the utility 
functions that comprise the model and determine the output 
are not easily manipulated to answer DM's various "what if" 
questions. In addition, if the value functions developed do 
not accurately reflect the actual Pareto optimal efficiency 
frontier, the output results will reflect the inaccuracy but 
without the knowledge of the DM. As one author has remarked, 
"For all its outward appearance of mathematical precision, 
MAUT is highly subjective, is not intuitive, and presents 
many difficulties in practice." [Ref. 30, pp. 256-257] 
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VIII. THE GOAL PROGRAMMING MODEL 



With this chapter the introduction to the primary, 
important multicriteria decision models for R&D project 
selection is completed. The goal programming (GP) method 
will be explained here through a description of the 
formulation procedures. This will be followed, as in the 
preceding three chapters with a summary of the strengths and 
limitations of GP for solving multicriteria problems such as 
those facing the Army's R&D community. 

A. INTRODUCTION TO GOAL PROGRAMMING 

The Goal Programming (GP) technique for supporting 
multiattribute decision making is a modification of linear 
programming, using mathematics to spread limited resources 
(time, manpower, funding, etc.) over multiple objectives. 
Abraham Charnes and William Cooper are frequently credited 
with first introducing this method in 1961. [Ref. 30] 

Because of its longevity, many complete formulations of 
the GP model for solving the R&D selection problem in the 
public sector are available for further study (see, for 
example, References 36, 44, and 45) . Nevertheless, in their 
1983 survey of 29 major "Fortune 500" industrial firms, 
Liberatore and Titus found only four of the 40 respondents 
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indicated that they were even aware of GP and none indicated 
that they used the technique to support their R&D project 
selection decisions. [Ref. 21] 

B. GOAL PROGRAMMING MODEL DESCRIPTION 

With linear programming the DM seeks to optimize a 
single objective given the constraints imposed by the 
environment in which the decision is made. It is very widely 
used, for example, in scheduling machine operating hours for 
various jobs. This is a well known example, where the 
constraints might be time and materials with the objective 
of maximizing production output to meet demand. 

With GP on the other hand, the object is to satisfy a 
number of competing objectives as completely as possible 
with the given resource constraints. As in our production 
example, we might set maximum production output as a goal 
along with say, minimizing operator overtime (accounting for 
worker safety and fatigue as well as expenses), minimize 
machine set-up times (caused by switching from one type of 
job to another) , and minimize finished goods inventory 
stack-up at the next work station. [Ref. 22] 

Each of these goals competes for the critical resources 
of time and materials which must be conserved while filling 
each of the competing demands as completely as possible. 
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This is a classic GP problem. Because of this capacity for 
handling multiple objectives, GP has been established as one 
of the dominate, if not the predominate MCDM. 

C. THE GOAL PROGRAMMING MODEL FORMULATION PROCESS 

The goal programming technique is powerful because once 
the problem is formulated into its linear objectives and 
constraints, a sound feasible solution can be obtained for 
the DM which is intuitive and logically supportable. The 
model is also appealing because it is flexible. 

Goals may be stated as absolute priorities, as 
objectives to be sought in some ranking by priority, or 
interval goal constraints can be specified with over or 
under achievement boundaries. Although the DM may state 
preferences between competing objectives as with AHP or 
MAUT, there is no requirement to do so. [Ref. 22] 

Weighting constants are not used in GP. Instead the 
objective functions contain plus and/or minus deviation 
variables, which represent surplus or slack resources that 
can be applied to competing objectives in the formulation. 

A negative deviation variable represents a floor limit set 
by the DM, while a positive deviation variable is a ceiling. 
Both plus and minus deviation variables (the dj*" factors in 
equation 7-3, of Figure 8.1) in one objective function 
designate an interval for required fulfillment. If the 
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objective is absolute, no deviation variables are included 
in that objective function. [Ref. 44] 

If absolute priorities are established for more than one 
objective in the formulation, the goals must be ranked in 
the DM's order of importance. Preferences can be expressed 
in many ways, but AHP is often recommended if a number of 
tradeoff comparisons must be determined. [Ref. 22] 

Once linear objective functions are constructed from the 
DM preferences, the GP technique uses a preemptive goal 
algorithm to maximize each objective. This means that lower 
importance goals are only considered after all higher 
objectives are fulfilled. Priority coefficients (the P k 
factor in equation 7-2 in Figure 8.1) can also be 
established to indicate priorities where slack or surplus 
resources should be assigned by DM preference. 

Find. . .X- {x 1 ,x 2 ,x i . . . ,x n ) (7-1) 

Maximize . . . Q='£ ) P k -f (dj , dj) (7-2) 
a^Xj+dj-dj^Cji 7-3) 

Figure 8 . 1 Example Goal Programming 

Formulation 

Figure 8.1 contains a generic GP formulation that 
includes each of the elements discussed so far. In this 
example formulation, the Xj factors are the project 
alternatives, the a^ is a coefficient value associated with 
a particular project (a quantity of resource j for 
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alternative i, for example), and Cj is the goal or 
objective value for constraint j. [Ref. 30, pp . 289-290] 

As this explanation may indicate, formulation of a 
multiple objective decision problem with competing 
priorities, and constraints into a mathematical model is no 
simple task. Goal programming is a recipe for transforming a 
problem, such as R&D project selection, into a model from 
which a solution can be derived. 

To summarize, there are three basic steps in the 
formulation process: 

• Determining the aspiration levels for goal constraints 

• Determining the slack or surplus allowed for each 
objective and the priority (if any) by which each is to 
be allotted to competing objectives, and 

• Development of the objective functions. 

D. ASSESSMENT OF THE STRENGTHS, LIMITATIONS, AND WEAKNESSES 
GOAL PROGRAMMING MODEL 

Although formulation of the specific GP model itself can 
be formidable when each step is straight forward, there are 
two assumptions that must be dealt with even before 
formulation can begin. First, each objective function 
derived will be linear and second, all goal constraints must 
be quantifiable for mathematical formulation. 

Both of these assumptions must be understood and agreed 
to by the DM before the objectives can be quantified and 



69 



formulated into a value function. Take for example an 
objective, "to minimize program risk." 

Development risk should decrease over time, as 
complications are worked out of the design approach. 

Schedule risk, on the other hand, may increase or decrease 
several times as the project continues, but it will most 
certainly not remain linear. In either case assumed 
linearity of the risk factor may or may not be significant 
enough to invalidate the model's output. After all, it is 
only a model. But generalizations of this kind can adversely 
effect DM confidence in the output reliability, thereby 
virtually invalidating the entire modelling effort. 

[Ref. 45] 

As for quantification, quantifying constraints like 
time, funding levels, personnel and materials is fairly 
routine in a GP formulation. If, however the DM's objectives 
are to "maximize benefit, capabilities, or value of the 
research efforts", GP alone will not suffice. [Ref. 46] 

One way to quantify the intangibles is to employ other 
techniques, such as AHP, to quantify qualitative factors and 
use the output as input to GP so that the problem can be 
formulated. An example of combining the two procedures was 
conducted for the U.S. Army Strategic Defense Command in 
1987 to determine R&D project selection given the program's 
multiple goals and constraints. [Ref. 36] 
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One important strength of this model is its ability to 
handle a vast number of constraints and objectives to 
evaluate numerous alternatives. In Dr. James Ignizio's work 
for the Army on R&D project selection, his goal programming 
code handled problems with over 100 variables, over 20 
constraints, and from three to five objectives [Ref. 

44, p. 271], Captain Anderson obtained reliable output for 
the Army Strategic Defense Command using his formulation of 
five objectives and 35 constraints to evaluate 35 
alternative projects [Ref. 36, pp. 49-59]. However, there is 
a limit. 

In their GP work, Choo, Wedley and Lam reported that 
their model "exploded" (a computer term for an endless loop 
which yields no feasible solution) with any evaluation of 
over 53 alternatives. The authors also pointed out however, 
that this figure was a matter of the software program used, 
the number of priority coefficients used in the objectives, 
and the extensive outcome probabilities used for each 
alternative. [Ref. 47] 

Finally, some researchers have argued effectively that 
the assessment of the DM preferences, priorities, and 
deviation variables in the GP formulation process is 
actually a search for the Pareto optimal alternative, as 
described in chapter VII. They point out that ignoring this 
fact can lead to suboptimal results. Halme and Korhonen, for 
example, develop the theory (with proofs) for incorporating 
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utility theory into the GP analysis in their 1988 work on 
multiple objective optimization problems. [Ref. 48] 

These references to research on the integration of 
several models is added to alert the reader. Overlap between 
the dominate MCDMs may be encountered in actual decision 
applications. This research effort indicated that successful 
applications blended the strengths of at least two models. 
Although this strategy may increase formulation time, the 
results have been encouraging. 

Goal programming is considered the first true MCDM . It 
has been evaluated and assessed widely, particularly 
overseas, over the past three decades. Almost without 
exception, the response to its results for improving the 
quality of decisions is overwhelmingly positive. However, 
the results of Titus and Liberatore's survey of top 
management from 29 "Fortune 500" companies in 1983 revealed 
that not only did American business not use this technique 
for R&D management, over 85 percent indicated that they were 
unaware of it. This indicates a clear disconnect between the 
management science model developers and the managers who are 
the users of these techniques. [Ref. 21] 
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IX. CONCLUSION AND SUMMARY 



There are many quantitative techniques available to 
aid the decisionmaker (sic) in arriving at a choice that 
has a high probability of meeting his objective.... The 
military decisionmaker must consider the ideas and 
approaches that decision technology has to offer. [Ref. 

49] U.S. Army War College, Army Command and Management , 
August 1991 

A. CONCLUSION 

The decision models presented in the last four chapters 
represent the state-of-the-art in American management 
science efforts to assist decision makers solve 
multicriteria decision problems. None of them will replace a 
decision maker or diminish the responsibility for the 
decision made. Each of them is after all, a representation, 
a model of a non-trivial decision process. They help to 
expand the bounds of rationality for those involved in the 
decision by prompting an analysis of as much of the 
available information as the decision maker desires. 

Each of the four modeling techniques represents a 
slightly different view of how complex decisions are 
structured to get a handle on and resolve the critical 
issues. For the Army Research, Development, and Engineering 
Centers, the problem is how to allocate limited resources 
over an ever widening range of technological opportunities 
to achieve each of their goals to the greatest extent 
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possible. This is a common problem that many of the authors 
and researchers identified herein have been working on for 
business, industry, and government for several decades. 

These models are not static, but are continually being 
reshaped to meet the needs of the user community. The users 
are international in scope, from Europe to the Pacific Rim 
and Australia. In fact, the models introduced in this thesis 
seem to be enjoying their greatest acceptance and 
application in Europe and Japan, according to the work at 
the International Conference on Multiple Criteria Decision 
Making [Ref . 39] . 

Decision theory in general and MCDM in particular will 
grow increasingly important to decision makers as the models 
become more sophisticated and user oriented. In 1983, The 
Harvard Business Review quoted Robert Newman of General 
Electric : 

Within 10 years, decision theory, should occupy the same 
role for the manager as calculus does for the engineer 
today. The engineer of Roman times was unaware of 
calculus, but he could make perfectly good bridges. 
However, he could not compete today, even in bridge 
building, let alone in astro-engineering. Management 
today is at the stage of Roman engineering. Needless to 
say, managers will still use specialists, just as 
engineers use heat transfer experts. [Ref. 50, p.149] 

This was not exceedingly prophetic, but rather a statement 

of the obvious. Decision models may not be as important as 

calculus yet, but their potential benefits for improving 

decision quality are enormous. 
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B. SUMMARY 



For the Army research and development project selection 
problem, MCDM hold a great deal of promise. With increased 
pressure from Congress for the Department of Defense to 
assure America's technological defense superiority in the 
face of dramatically reduced defense spending, there will be 
increased scrutiny and accountability for the Army's R&D 
resource commitment decisions. Multicriteria decision models 
offer a means to develop logically supportable solutions to 
this issue. 

The Army decision maker will rarely know with absolute 
certainty which R&D alternatives hold the greatest promise 
for achieving the Army investment strategy objectives. The 
final choice will likely be something of a compromise, a 
tradeoff among judgments based on quantitative and 
qualitative factors and valued expert opinion on potential 
benefits, risks, costs, and payoffs. 

These intuitions from competing stakeholders must be 
assessed on a level basis, as free as possible from human 
biases, and in keeping with organizational values. Multi- 
criteria decision models can incorporate these elements, 
organize the decision criteria, and quantify subjective 
opinion regarding alternatives' relative merits so that the 
DM can make a rational choice on the basis of available 
data . 
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Furthermore, the limits of human cognitive skills 
introduce additional inefficiencies of their own. 
Multicriteria decision models can bring structure to these 
non-trivial decisions and systematically aid the decision 
maker to overcome the random errors common to multicriteria 
problems of choice. Clearly if a decision problem can be 
effectively modeled, then the quality of the decision can be 
improved . 

In sum, the quality and efficiency of the Army R&D 
resource commitment must increase. The decisions to commit 
resources to achieve organizational goals and execute 
mandates are perhaps the single most important function of 
any leader. Nevertheless, precious little time can be 
dedicated to a comprehensive analysis of all of the factors 
that support these decisions. Multicriteria Decision Models 
clearly offer an attractive means for resolving this 
dilemma . 
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